This tutorial was built for people who wanted to learn the essential tasks required to process text for meaningful analysis in R, one of the most popular and open source programming languages for data science. At the end of this tutorial, you’ll have developed the skills to read in large files with text and derive meaningful insights you can share from that analysis. You’ll have learned how to do text mining in R, an essential data mining tool. The tutorial is built to be followed along with tons of tangible code examples. The full repository with all of the files and data is here if you wish to follow along.

Searching for a job using R?  Check out our list of R Interview Questions first!

If you don’t have an R environment set up already, the easiest way to follow along would be to use Jupyter with R. Jupyter offers an interactive R environment where you can easily modify inputs and get the outputs demonstrated rapidly so you can rapidly get up to speed on text mining in R.

Natural languages (English, Hindi, Mandarin etc.) are different from programming languages. The semantic or the meaning of a statement depends on the context, tone and a lot of other factors. Unlike programming languages, natural languages are ambiguous.

Text mining deals with helping computers understand the “meaning” of the text. Some of the common text mining applications include sentiment analysis e.g if a Tweet about a movie says something positive or not, text classification e.g classifying the mails you get as spam or ham etc.

In this tutorial, we’ll learn about text mining and use some R libraries to implement some common text mining techniques. We’ll learn how to do sentiment analysis, how to build word clouds, and how to process your text so that you can do meaningful analysis with it. 

R is succinctly described as “a language and environment for statistical computing and graphics,” which makes it worth knowing if you’re dabbling in the data science/art of statistics and exploratory data analysis. For data scientists who are working with statistical analysis, knowing R is a must. R has a wide variety of useful packages for data science and machine learning.

Here, we’ll focus on R packages useful in understanding and extracting insights from the text and text mining packages.

In this tutorial, we will be using the following packages:

You can install the aforementioned packages using the following command:install.package(“package name”) 

Before we dive into analyzing text, we need to preprocess it. Text data contains white spaces, punctuations, stop words etc. These characters do not convey much information and are hard to process. For example, English stop words like “the”, “is” etc. do not tell you much information about the sentiment of the text, entities mentioned in the text, or relationships between those entities. Depending upon the task at hand, we deal with such characters differently. This will help isolate text mining in R on important words.

Haotian Wu

Data Scientist at RepTrak

Read Story

Pizon Shetu

Data Scientist at Whiterock AI

Read Story

Leoman Momoh

Senior Data Engineer at Enterprise Products

Read Story

A word cloud is a simple yet informative way to understand textual data and to do text analysis. In this example, we will try to visualize Hillary Clinton’s Emails. This will help us quantify the content of the Emails and help us derive insights and better communicate our results Along the way, we’ll also learn about some data preprocessing steps that will be immensely helpful in other text mining tasks as well. Let’s start with getting the data. You can head over to Kaggle to download the dataset. 

Let’s read the data and learn to implement the preprocessing steps.

[code lang=”r” toolbar=”true” title=”Reading data in with R”]library(RSQLite) db